A Framework for Accelerating High - dimensionalNN

The performance of nearest neighbor (NN) queries degrades noticeably with increasing di-mensionality of the data. This stems not only from reduced selectivity of high-dimensional data but also from an increased number of seek operations during query execution. We propose a new framework to transform NN queries into at most two range queries. This is achieved by rst estimating the NN-radius, performing a range query with this radius, and (if not enough NNs were found) estimating the NN-radius again and performing a second range query. Since range queries know the set of pages to be read in advance, a page read scheduler can be employed to minimize read costs. This scheme guarantees that the query cost is bounded from above by the cost of 2 linear scans over a subset of the data pages, while typically resulting in costs well below a single scan. Our framework can be instantiated with diierent range query index structures, radius es-timators, and page read schedulers. We present one possible implementation of the radius estimator based on sampling and the fractal dimensionality of data. Since the overall cost depends on the quality of this radius estimate, we examine it analytically and experimentally. Our analysis for uniform data indicates that the estimation error is below 14% and that errors in the fractal dimensionality estimation have only minor impact on the overall cost. Experiments on synthetic and real datasets show that our new technique reduces the I/O cost during NN queries by a factor of up to 15 for an R ?-tree based index structure and up to 5 for an X-tree-like bulkloaded index structure.